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METHOD AND APPARATUS FOR CONTINUOUS FOCUS 
AND EXPOSURE IN A DIGITAL IMAGING DEVICE 



5 FIELD OF THE INVENTION 

The present invention relates generally to digital photography and more 
specifically to continuous focus and exposure techniques for digital imaging devices. 

BACKGROUND OF THE INVENTION 
10 Digital imaging devices such as digital cameras typically include some kind 

of automatic focus and exposure feature. At one extreme, some digital cameras 
perform coarse exposure calculations to keep the image within reasonable exposure 
range, deferring fine focus and exposure calculations until the user presses the shutter 
button to the intermediate (SI) position. The result is considerable delay (often 
15 greater than one second) after the shutter button is depressed to the image capture (S2) 
position before the final image is captured. Although such an approach minimizes 
battery power consumption, it introduces shutter lag. At the opposite extreme, other 
digital cameras calculate focus and exposure continuously, even when the user is not 
intending to capture an image imminently. This approach consumes more battery 
20 power but shortens shutter lag. Still other digital cameras perform a coarse focus 
update using low-resolution frames when a lighting change is detected. Such an 
approach still requires a fine focus adjustment at SI, again resulting in shutter lag. 

It is thus apparent that there is a need in the art for an improved method and 
apparatus for continuous focus and exposure in a digital imaging device. 

25 
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SUMMARY OF THE INVENTION 
A method for automatically maintaining focus and exposure settings in a 
digital imaging device is provided. A digital imaging device to carry out the method 
is also provided. 

5 Other aspects and advantages of the present invention will become apparent 

from the following detailed description, taken in conjunction with the accompanying 
drawings, illustrating by way of example the principles of the invention. 

BRIEF DESCRIPTION OF THE DRAWINGS 
10 Fig. 1 A is a functional block diagram of a digital camera in accordance with 

an illustrative embodiment of the invention. 

Fig. IB is a functional block diagram of an imaging module of the digital 
camera shown in Fig. 1 A in accordance with an illustrative embodiment of the 
invention. 

15 Fig. 1C is a functional diagram of a memory of the digital camera shown in 

Fig. 1A in accordance with an illustrative embodiment of the invention. 

Figs. 2A-2C are a flowchart of the operation of the digital camera shown in 
Fig. 1 A in accordance with an illustrative embodiment of the invention. 

Fig. 3 is an illustration of a portion of a Bayer pattern associated with an 
20 imaging sensor having an odd number of fields in accordance with an illustrative 
embodiment of the invention. 

DETAILED DESCRIPTION OF THE INVENTION 
An effective trade-off between battery power consumption and shutter lag may 
25 be achieved by monitoring low-resolution frames to detect a scene change and 
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performing focus and exposure calculations based on high-resolution frames, when a 
scene change has been detected. These operations may be associated with a 
continuous focus and exposure mode of a digital imaging device. The continuous 
focus and exposure mode may be activated manually or automatically in a variety of 

5 ways. Also, a variety of different formats may be employed for the high-resolution 
frames that are used to adjust focus and exposure. Scene analysis to detect scene 
changes may be simple (e.g., detection of a change in aggregate luminance) or 
complex (e.g., detection of a single moving element in an otherwise static scene to 
avoid unnecessary adjustments). Further, such a continuous focus and exposure mode 

10 may be operated with the live view feature (low-resolution video of the current scene) 
of the digital imaging device enabled or disabled. 

The remainder of this detailed description presents the techniques and 
principles of the invention in the context of a digital camera, but those techniques and 
principles are applicable to any digital imaging device, including a digital camcorder 

15 having a still-picture capture mode. 

Fig. 1A is a functional block diagram of a digital camera 100 in accordance 
with an illustrative embodiment of the invention. In Fig. 1 A, controller 105 
communicates over data bus 110 with imaging module 115, memory 120, activation 
subsystem 125, display buffer and control logic 130, input controls 135, and shutter 

20 button 140. Display buffer and control logic 130 is in turn connected with display 
145. Display 145 may be, for example, a liquid crystal display (LCD). Optical 
system 150 produces optical images that are converted to digital images by imaging 
module 115. Input controls 135 may comprise navigational buttons, an input control 
(e.g., a button or switch) to activate a continuous focus and exposure mode in digital 

25 camera 100, and other input controls to control the operation of digital camera 100. 
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Shutter button 140 may have an intermediate position (SI) and an image capture 
position (S2). 

Fig. IB is a functional block diagram of imaging module 1 15 in accordance 
with an illustrative embodiment of the invention. Imaging module 115 may comprise 
5 an imaging sensor 155 (in this example, a CCD sensor array), a timing 

generator/analog front end (TG/AFE) 160, and a digital signal processor (DSP) 165. 
As indicated in Fig. 1 A, imaging module 1 15, via DSP 165, may communicate 
directly with controller 105 in some embodiments. As indicated in Fig. IB, both data 
and control signals connect imaging sensor 155 and TG/AFE 160. Imaging module 

10 1 15 is configurable to produce both low-resolution frames (e.g., those associated with 
live view mode) and high-resolution frames as needed. 

Fig. 1C is a functional diagram of memory 120 in accordance with an 
illustrative embodiment of the invention. Memory 120 may comprise random access 
memory (RAM) 170, non-volatile memory 175, scene analysis logic 180, focus and 

15 exposure adjustment logic 185, and activation logic 190. In some applications, non- 
volatile memory 175 may be of the removable variety (e.g., a secure digital or multi- 
media memory card). 

Scene analysis logic 1 80 analyzes low-resolution frames from imaging module 
1 15 to detect a scene change. In one embodiment, scene analysis logic 180 sums the 

20 absolute value of pixel differences between two or more low-resolution frames to 
detect the scene change. This technique, well known in the image compression art 
(e.g., MPEG-4), typically involves blocking the digital frames into 8-pixel x 8-pixel 
or 16-pixel x 16-pixel blocks. In the image compression context, absolute pixel 
differences between frames may be summed for each of a number of spatial offsets in 

25 a neighborhood surrounding each block to produce a motion vector indicating how the 
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camera or the scene shifts spatially from one frame to another. In the context of the 
instant invention, a motion vector may not be required in some embodiments. In 
those embodiments, a simple metric comprising the sum of the absolute value of the 
pixel differences between low-resolution frames at zero spatial offset may suffice. 
5 For example, when the sum of the absolute value of the pixel differences exceeds a 
predetermined threshold, a scene change may be assumed to have occurred. In other 
embodiments, mean squared error may be used in computing the metric instead of the 
absolute value of pixel differences. 

In another embodiment, scene analysis logic 180 converts the low-resolution 

10 frames to the well-known (Y,C b ,C r ) color space (luminance and two chrominance 
difference components). Using the Y (luminance) component, scene analysis logic 
1 80 may compute an aggregate scene brightness for each low-resolution analysis 
frame. The aggregate scene brightness may be computed from the luminance 
component by a number of well-known methods, some of which may include the use 

15 of a histogram. By comparing the aggregate scene brightness of two or more low- 
resolution frames, scene analysis logic 180 can detect a scene change. 

In yet another embodiment, scene analysis logic 180 may be configured to 
detect a single moving element in an otherwise static scene (e.g., a person or object 
moving laterally with respect to digital camera 100). Although a literal scene change 

20 occurs in such a case (i.e., some pixels change from frame to frame), there is no need 
to adjust focus and exposure because the focus distance (the distance from digital 
camera 100 to the single laterally moving element) has not changed. Configuring 
scene analysis logic 1 80 to detect this situation prevents unnecessary automatic fine 
focus and exposure adjustments in digital camera 100. Techniques for identifying a 

25 single moving element in an otherwise static scene are well known in the image 
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processing art, especially in military and surveillance applications. For example, the 
fraction of pixels that change from frame to frame may be determined. A high 
percentage of unchanged pixels may indicate a single moving element in the scene. 
The presence of a single moving element may be confirmed through use of the 
5 techniques described above for computing a motion vector. 

The low-resolution frames analyzed by scene analysis logic 180 do not have to 
be successive. For example, scene analysis logic 180 may analyze low-resolution 
frames that are separated by one or more intervening frames. 

Focus and exposure adjustment logic 185 adjusts focus and exposure based on 
10 high-resolution frames when scene analysis logic 180 has detected a scene change. 
Techniques for performing fine focus and exposure adjustments based on high- 
resolution frames are well known in the digital camera art. 

Activation logic 190 may activate a continuous focus and exposure mode in 
digital camera 100 in response to a predetermined signal from activation subsystem 
15 125. Activation subsystem 125 and its associated output signal may take on a variety 
of forms. 

In one embodiment, activation subsystem 125 comprises an attitude sensing 
subsystem that detects how digital camera 100 is being held by a user. For example, 
the attitude sensing subsystem may comprise one or more accelerometers or a 
20 gyroscope. When digital camera 100 is being held in a predetermined attitude 

associated with picture taking (e.g., the axis of optical system 150 is approximately 
parallel to the ground), activation subsystem 125 reports the condition, and activation 
logic 190, in response, may activate the continuous focus and exposure mode of 
digital camera 100. 
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In another embodiment, activation subsystem 125 may comprise a tactile 
sensing subsystem that detects when a user is lightly touching shutter button 140. 
Such tactile sensing subsystems are well known in the art and may be based on a 
detected change in resistance or capacitance in a circuit. When a user lightly touches 
5 shutter button 140, activation logic 190 may activate the continuous focus and 
exposure mode in response to a signal from the tactile sensing subsystem. 

In another embodiment, activation subsystem 125 may comprise an optical 
sensor that detects when a user places a finger on or near shutter button 140. When a 
user places a finger near shutter button 140, activation logic 190 may activate the 
10 continuous focus and exposure mode in response to a signal from the optical sensor. 

In yet another embodiment, activation logic 190 may activate the continuous 
focus and exposure mode in response to shutter button 140 being depressed to the SI 
position. 

In still another embodiment, activation logic 190 may activate the continuous 
15 focus and exposure mode in response to actuation of an input control 135 (e.g., a 
continuous focus and exposure mode button), as noted above. In a variation of this 
embodiment, the continuous focus and exposure mode may be selected from a menu 
on display 145 of digital camera 100 with the aid of input controls 135. 

The divisions among activation subsystem 125, scene analysis logic 180, focus 
20 and adjustment logic 185, and activation logic 190 are purely functional. In general, 
these functions may be separated or combined in ways other than that shown in Figs. 
1 A and 1C and may be implemented in hardware, firmware, software, or any 
combination thereof. In one embodiment, scene analysis logic 180, focus and 
exposure adjustment logic 185, and activation logic 190 are stored program 
25 instructions that may be executed by controller 105. 
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Figs. 2A-2C are a flowchart of the operation of digital camera 100 in 
accordance with an illustrative embodiment of the invention. Point "A" is reached 
when activation logic 190 has activated the continuous focus and exposure mode of 
digital camera 100. At 205, imaging module 115 captures low-resolution frames, and 
5 focus exposure and adjustment logic 185 performs a coarse automatic exposure 
adjustment. If the live view mode of digital camera 100 is enabled at 210, the low- 
resolution frame is shown on display 145 at 215. Live view mode typically displays 
low-resolution frames at 30 frames per second. At 220, the low-resolution frame may 
be stored in a buffer (e.g., some portion of memory 120). Scene analysis logic 180 

10 analyzes two or more stored low-resolution frames at 225 to detect a scene change. If 
a scene change is detected at 230, control proceeds to point "B" in Fig. 2B. 
Otherwise, control returns to point "A." Steps 205 through 230 in Fig. 2A may be 
termed a "Monitor & Live View Sequence" in which low-resolution frames are 
captured and analyzed to determine whether or not a scene change has occurred. 

15 Fig. 2B shows optional steps that may be performed to further enhance the 

performance of the continuous focus and exposure mode. At 235, focus and exposure 
adjustment logic may perform coarse focus and exposure adjustments based on low- 
resolution frames. At 236, imaging module 115 captures a low-resolution frame. If 
live view is enabled at 237, the low-resolution frame is shown on display 145 at 238. 

20 This process is repeated until the test at 240 is satisfied, indicating that coarse focus 
has been achieved. Once coarse focus has been achieved, control proceeds to point 
"C" in Fig. 2C. 

If live view mode is enabled at 245 in Fig. 2C, a low-resolution frame 
captured at 250 is presented on display 145. Otherwise, control skips to 255, where a 
25 high-resolution focus frame is captured. Further details regarding high-resolution 
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frames are provided later in this detailed description. At 260, optical system 150 may 
be adjusted by a single focus step. If the best focus has been achieved at 265, focus 
and exposure adjustment logic 185 may perform a fine auto exposure adjustment at 
270, after which control returns to point "A." Otherwise, control returns to 245, and 
5 the fine focus and exposure loop continues until convergence is achieved. Steps 245 
through 270 may be termed a "Focus & Exposure Sequence" that is activated when a 
scene change is detected by the "Monitor & Live View Sequence." 

If shutter button 140 is depressed to the image capture position (S2), a final 
fine focus and exposure adjustment may be performed, if necessary, and imaging 
10 module 115 may capture a final digital image. 

The high-resolution frames upon which focus and exposure adjustment logic 
185 bases its focus and exposure adjustments may be in any of a variety of formats. 
For example, the high-resolution frames may be full-resolution readouts from imaging 
sensor 155. In other embodiments, the high-resolution frames may be in the Video 
15 Graphics Array (VGA) format, the Super VGA (SVGA) format, the 720P format 
(intermediate between SVGA and high-definition), or any other suitable high- 
resolution format. 

In one embodiment, a different kind of high-resolution frame may be 
employed that takes advantage of the properties of CCD imaging sensors 155 having 
20 an odd number of fields per frame. Fig. 3 is an illustration of a portion of a Bayer 
pattern associated with a CCD imaging sensor 155 that has three fields per frame in 
accordance with an illustrative embodiment of the invention. The rows of imaging 
sensors associated with one of the three fields are indicated by the arrows on the left 
side of the figure. Reading out the single field indicated by the arrows can produce a 
25 coherent image because the readout includes red, green, and blue color information. 
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Therefore, the high-resolution frames may be single-field readouts from an imaging 
sensor 155 having an odd number of fields. 

The foregoing description of the present invention has been presented for the 
purposes of illustration and description. It is not intended to be exhaustive or to limit 

5 the invention to the precise form disclosed, and other modifications and variations 
may be possible in light of the above teachings. The embodiments were chosen and 
described in order to best explain the principles of the invention and its practical 
application to thereby enable others skilled in the art to best utilize the invention in 
various embodiments and various modifications as are suited to the particular use 

10 contemplated. It is intended that the appended claims be construed to include other 
alternative embodiments of the invention except insofar as limited by the prior art. 
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